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ABSTRACT 

Marr has emphasized the difficulty in understanding a biological system or its components without 
some idea of its goals. In this paper, a preliminary goal for color vision is proposed and analyzed. That 
goal is to determine where changes of material occur in a scene (using only spectral information)! This 
goal is challenging for two reasons. First, the effects of many processes (shadowing, shading from sur- 
face orientation changes, highlights, variations in pigment density) are confounded with the effects of 
material changes in the available image intensities. Second, material changes arc essentially arbitrary. 
We are consequently led to a strategy of rejecting the presence of such confounding processes. We 
show there is a unique condition, the spectral crosspoint, that allows rejection of die hypothesis that 
measured image intensities arise from one of the confounding processes. (If plots are made of image 
intensity versus wavelength from two image regions, and the plots intersect, we say that there is a 
spectral crosspoint.) We restrict our attention to image intensities measured from regions on opposite 
sides of an edge because material changes almost always cause edges. Also, by restricting our attention 
to luminance discontinuities, we can avoid peculiar conspiracies of confounding processes that might 
mimic a material chage. Our crosspoint conjecture is that biological visual systems interpret spectral 
crosspoints across edges as material changes. A circularly symmetric operator is designed to detect 
crosspoints; it turns out to resemble die double-opponent cell which is commonplace in biological 
color vision systems. .... . 
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Figure 1. Notation. A) X and y are two regions in an image. B) The available 
information is continuous functions of image intensity versus wavelength, /x(X) and 
/y(X), from regions X and Y respectively. C) Discrete representation of this information 
is possible by taking a number (here, three) of spectral samples (narrowband in this 
illustration, at wavelengths Xi, X2, and X3). 

Our goal is thus similar to the one first proposed by Land (1977), but more modest. Land was 
concerned, as we are, with computing information about surface properties of materials in a scene. 
His objective was to determine the reflectance of regions using only the available intensity informa- 
tion. Land reached his objective by assuming a greatly simplified Mondrian world— a flat, shadowless 
world composed of regions of uniform reflectance. We prefer to deal with the natural world in its 
entirety without unnecessary simplification, and instead limit our objectives. In this way we can 
exploit the regularities in the world as assumptions or constraints in the solution of our problem. The 
simple goal we will analyze, to reiterate, is just to determine when changes in an image arise from 
one material's bordering (or occluding) another in a scene. We shall sec below that the problem to be 
solved in achieving our modest goal is still formidable. 
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GLOSSAR VfcF SYMBOLS 



X!f— material changes 
&— -shadow (point source) 
/^—highlight or gloss 
6— change in surface orientation 
^—change in pigment density 
S^—shadow-with penumbra (extended source) 
u,impit— monotonic change in surface orientation 
II. Quantities measured fa Image 
/x(M— the continuous function of image intensity versus wavelength measured in a region X in an 



lix — a discrete spectral sample at wavelength X* 

III. Continuous Scene Variables 

p(X)— the albedo or spectral reflectance function of a material. 

£b(*)— the diffuse component of the illumination. 

£s(M— the synthetic component of the illumination. 

E{ty— the illuminam taken as a whole (when components are irrelevant). 

a—the penumbra constant. 

*yx— the thickness of pigmentation in region X. 

IV. Discrete Scene Variables 

If QxW is some function of wavelength in region X t men Q iX is a discrete sample of that 
function, at wavelength X„ from region X. 

TaWe 1. A glossary of symbols. 



Tabic 1. A glossary of symbols. 

2 The Scope of the Problem 



,/^N 



The problem in achieving the proposed goal of finding changes of material (hereafter denoted by 

the symbol M 1 ) is that any given image intensity can arise in many different ways, depending on 

the particular processes in effect. Thus given a single intensity value in the image, it is generally not 

possible to decide which of the many possible events in the world produced it. It can be said that the 

act of imaging a scene, or projecting it into two dimensions, confounds the effects of material changes, 

our interest here, with the effects of other processes. These confounding events include shadows, 

surface orientation changes, highlights, and variations in pigment density. Furthermore, the quest 

for recovering M (material) changes from image intensities must succeed under a range of spectrally 

different illuminants. A system that only worked properly given an illuminant having the spectral 

composition of the mid-day sun would be very limited. At dawn and dusk, when sunlight reddens, the 

system might fail. 

hlie circumflex accent is used in this paper to denote abstract processes: it is intended to keep discussion about processes 
distinct from talk about simple variables or functions. 
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We can now summarize the problem that early color vision faces, if its goal is to discover material 
changes: 

The problem in determining where changes of material occur in a scene is that in the available image 
intensity values, the effects of many processes may be confounded 

In the remainder of section 2, a notational scheme will be developed. In section 3, a general outline 
of the solution will be presented. 

Notation. A notational scheme is shown in figure L Regions in the static image will be denoted by 
X and Y. (The time variable will be ignored.) These letters will also be used to refer to regions in a 
scene which are the inverse projections of the image regions. The context will make clear the correct 
referent. The image intensity measurable in region X (or Y\ say, as a function of wavelength, will be 
denoted by the function /*(X) (or 7y(X)). Note that I x [\) (or Iy(\)) represents all the information 
available from region X: it is the continuous spectral distribution of image intensity from X (or Y) y as 
shown in the middle graph in figure 1. In section 5, we will refer to discrete approximations of 7x(X), 
which can be generated by sampling the image intensity from X at spectral points (X b . . . , X n ), as 
shown in the bottom graphs in figure 1. In discrete computations, I jX will denote the image intensity 
measured from region X at X^. 

Since many symbols are used in this discussion, a glossary is presented in table 1. 

3 The Theme of the Solution: A Negative View 

3.1 The Strategy 

Given two spectral energy distributions 7 A (X) and 7y(X), how might we determine whether they 
arise from an M (material) change? Little can be said about the spectral nature of M changes; they 
are essentially arbitrary. No simple equations can relate image intensities from two regions composed 
of different materials. The major confounding processes— shadows (hereafter denoted 5), highlights 
(#), surface orientation changes (6), and changes in pigment density (P)— however, produce lawful 
changes in the image. Simple equations can capture this lawfulness, as will be seen in section 4. 
Suppose that by examining image intensities from two regions we could eliminate the possibility that 
I s they arose from cither an 5, H, 6, or P change. Since we arc assuming the illuminant to be spectrally 

invariant in a neighborhood, we would like to conclude that the change was due to a material change. 
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A preliminary conjecture is thus proposed: 

When a difference between (he image intensities I x {^) andly(\) (taken from two image regions X 
and Y) does not arise from one of the confounding processes S, H, 6, orP, then this difference between 
X and Y is due to a difference in materials, orM. 

The conjecture above suggests a computational strategy of attempting to reject measurements of 
image intensities as arising solely from shadow (5), highlight (H\ surface orientation change (6), or 
pigment density change (P), the lawful confounding processes in a scene. Our rejection strategy will 
be correct only if there are no confounding processes other than the ones mentioned. But, as will be 
discussed later, if in the course of rejecting the presence of S, H, O, and P, we also reject the presence 
of any of a large class of other possible confounding processes, the strategy will be a powerful one and 
the conjecture will be useful for practical purposes. 

It is important to note that rejecting the presence of a lawful process is often much easier than ac- 
cepting it. As mentioned above, lawful processes are associated with equations. These equations relate 
quantities measurable in the image (constants in the equations) to scene properties (variables in the 
/"■■^ equations) which are not directly measurable. Typically, these scene property variables (reflectances, 

for example) are constrained to take values within a certain interval. As will be seen below, rather 
than attempting to solve a system of equations (subject to constraints on the values of variables), it 
is often computationally simpler to determine whether the system (with constraints) can be solved. 
In this sense, rejecting a solution (process) is easier than accepting one. By analogy, disproving a 
conjecture about number theory (some universally quantified equation or inequality) with a single 
counterexample is simpler than demonstrating the theoremhood of the conjecture. 2 

Here's a simple example. Consider the equations h — JK and h = J K , where J and K are 

variables, and h and I 2 are constants. If the variables J and K are constrained to have values greater 

than unity, then a simple test can be made that might determine that the system has no solution. 

Specifically, if min(/i,/ 2 ) < 1, then there is no solution that obeys the constraints 3 . Intuitively, 

2 Of course, rejecting the occurrence of a process or event is logically equivalent to accepting the occurrence of its 
negation. But usually either a process or its complement, and not both, can be characterized mathematically. So the 
logical equivalence breaks down in favor of practical considerations, such as describability. Intuitively, rejecting the 
presence of a shadow is child's play compared to accepting the presence of a nonshadow. How could the class of all 
(visually interesting) events-that-are-not-shadows possibly be characterized? That class is certainly a peculiar collection 
of odds and ends, including such diverse members as paths of fireflies and holes in the ground. Visual systems "don't 
care" if they're accepting processes or rejecting negations of processes; it only matters to us when we characterize what 
we think the system might be doing. 

^""^ 3 Clcarly the product of two numbers each greater than unity is greater than unity. And a number greater than unity 

raised to a power greater than unity is also greater than unity, 'lhe rejection condition niin(/ lt / 2 ) < 1 is just the 
contrapositive of the two statements above. 
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this decision about unsolvability is computationally easier than actually finding a solution. 

3.2 The Strategy Applied to some Confounding Processes 

Figure 2 illustrates intuitively how the rejection strategy will be applied. Each of the first three 
panels (O, H, P) show the effects (in a graph of image intensity versus wavelength) of a surface 
orientation change, a highlight, or a change in pigment density in a planar patch of a single material. 
(Shadow changes S are similar to highlights.) As will be shown below, the effect of a surface orien- 
tation change is to cut down the direct illumination by a constant fraction. Highlight or gloss is a 
situation of increased image intensity at all wavelength. (A shadow is the opposite.) Finally, pigment 
density changes, such as the variations seen in the grain of wood, can be characterized as several light- 
absorbing filters in sequence. The effect of a stack of filters is to reduce the available light according 
to a power relation. (The concentration of a dye dispersed in liquid affects transmitted light according 
to a power law more precisely.) What all these natural processes have in common is that they act 
to increase or decrease image intensities across wavelength. Violation of those sorts of displacements 
can be used to reject the presence of shadow, orientation change, gloss or highlight, or change in 
pigment density. Therefore, if two spectral functions of image intensities are not related by one always 
lying above the other, then the functions come from two regions that might be composed of different 
materials. The lower right portion of figure 2 illustrates such a situation, which is a candidate for a 
material change. 

In the next section we examine simple models of the confounding processes. In section 5, we 
address the problems: How many channels does a visual system require in order to reject the presence 
of particular processes in a scene? What sort of computations with available image intensities allow 
rejection of confounding processes? 

4 The Physics Behind the Scenes 



r^ 
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4.1 The Image Intensity Equation 



When light is reflected from a surface into the eye, the image intensity 7(X) depends on several 
factors. The surface properties of the object interact with the geometry of the viewing situation and 
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Figure 2. Three different relations are illustrated that can hold between two spectral functions of images 
intensities from regions X and Y. At top (left), a multiplicative relation (surface orientation change) is 
illustrated. At right is depicted an augmentative relation (highlight, for example). Bottom (left) is a power 
relation, typical of pigment density changes. At rigfu is a change that is not an increase or decrease of intensity 
across wavelength. Such a situation is a possible material change. 

the spectral nature of the light source to produce image intensity. 4 In the case of a matte surface 5 
these effects combine multiplicatively to yield the following image intensity equation: 



/-\ 



/(X) = p(X)E(X)[N.L]^^) 



(1) 



where p{\) is the reflectance or albedo of the surface, R(0, <f>) is the bidirectional reflectance distribu- 
tion function (see Horn & Sjoberg, 1979) which describes properties of the surface dependent on its 
orientation with respect to viewer (<f>) and light source (0), and [N ♦ L] is the angular relation between 
the surface normal, N, and the illuminant direction L. Any collection of multiple light sources is 

^technically, /(X) for a point x,y on the surface is called image irradiance. Image intensity takes into account other 
factors such as pupil size and luminance constants. These details are not important to the argument here, and will be 
omitted. 

5 For specular surfaces, a different image intensity equation holds. See Appendix I. 
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Figure 3. How shadow-with-penumbra is cast. A) Between the full lit and frilly shadowed regions, some 
proportion of the source can reach the object B) An (achromatic) image intensity profile is shown for the 
surface. 

equivalent to a single source (called here the "synthetic source") on unshadowed portions of surfaces 
(Silver, 1980), so regardless of the complexity of the illumination, a single function E(\), together 
with directions and 0, will characterize the direction and spectral nature of the illuminant for 
unshadowed surfaces. 

The effect of many natural processes that affect images can usually be characterized simply as ac- 
ting on one or more of the multiplicative factors in equation (1). For example, a shadow corresponds 
principally to a reduction of the illuminant E{\). Surface orientation changes correspond to to 
changes in the [N • L] term, and pigment density changes affect only the p(\) term. (Highlight or gloss 
requires that a specular term be added to equation (1).) All of the processes discussed above can occur 
in a region of a single material. 

We now proceed to examine how common natural processes other than material changes will affect 
the image intensity equation, and what these confounding processes have in common that allows their 
rejection en masse. 
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4.2 Shadows 

Suppose a shadow (S) falls on a surface composed of a single material, and furthermore, suppose 
that the only changes across the surface are these due to shadow. (Specifically, there will be no 
changes in surface orientation or variations in pigment density on the surface.) Then the shadow can 
be described by the following equations: 

IutW = [EsW + E D (\)]p(\) 

IskaM=E D (\) P (\) {2a) 

where p(X) is the albedo of the material. The image intensities, as functions of wavelength, from the 
lit and shadowed regions are I lit (\) and I s hade{^l respectively. E D {\) represents a diffuse component 
of illumination striking both lit and shadowed regions, and£s(X) is some additional synthetic source 
striking only the lit region. 6 

If the illuminant is an extended source, like the sun, there will be penumbrae (see figure 3), and a 
third equation must be included: 

J pe „(\) = [oEs{\) + E D {\)]p(\) (2b) 

where a is a constant between and 1, and I pen {\) is the image intensity (as a function of wavelength) 
in some region in the penumbra. (The symbol S pen will denote shadows with penumbrae.) 

4.3 Surface Orientation Changes 

Suppose that two image regions X and Y differ only in their surface orientations. That is, assume 

that X and Y receive the same illumination (no self-shading) and are composed of the same material. 

This process as been dentoed by the symbol 6. The patches X and Y that differ only in surface 

orientation have respective surface normals Nx and Ny which form angles X and 0y with the 

6 If light is available from the shadowed region, it arises from some collection of sources. According to Silver (1980), a 
collection of sources is equivalent to a single source provided that they all illuminate the same surface patch. We call 
this synthetic source E D (\). Furthermore, this diffuse synthetic source acts as if it lies in the direction of the viewer's 
eye (Horn, 1975). Next consider the lit region. Again, by Silver's proof, there is a single source that is equivalent to the 
illumination reaching the lit region. Let's call this source E S *(X), and its direction L s * Since we can see the lit region, 
we know that there is some component of L s * in the direction of the viewer. So we can perform a simple vector 
decomposition of the synthetic illuminant for the lit region into a component in the viewers direction (so this component 
is identical to the synthetic diffuse illuminant E n (\) above), and some other component Ii s {\) with a new direction 
L v . So in equations (2a) £/j>(K) abbreviates a product of a term representing the spectral flux of the illuminant, a 
bidirectional reflectance term, and a surface orientation term, with the illuminant direction L in the viewer's direction. 
Likewise, Es(X) stands for the product of of the same three terms above, but in some synthetic direction L s . 
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(synthetic) light source direction L, and angles <t> x and 4> Y with the viewer direction V. Using the 
image intensity equation (1), the expected image intensities from regions X and Y differing only in 
surface orientation are: 

/x(X) = p(M£(X)[Nx * L]R(0 x , 4> x ) 

MX) = p(X)B(X)[N y • L]R(e Y} <t> Y ) (3a) 

where again I x [\) and I Y (\) are the image intensities from regions X and Y as functions of 
wavelength. 

Both equations of (3a) have a pair of multiplicative terms not involving wavelength. Let (3 X = 
[Nx • L]R(0 X) <j> x \ and let f3 Y be similarly defined. Equations (3a) can now be simplified with this 
consolidation of constant factors: 

/x(X) = /MAMX) 

4.4 Changes in Pigment Density 

The reflectance of materials is determined primarily by the density of a pigment in some embed- 
ding layer of the material, and the thickness of the pigment layer. The embedding layer in leaves, 
for example, is cellulose, and the usual pigment is chlorophyll. Some surfaces may be unevenly 
pigmented. The grain of a wooden table provides a good example of a change in pigment density 
across a surface. This sort of change will be labeled P? 

Several different laws relate changes in pigment density to changes in reflectance. (See Judd & 
Wyszecki, 1963.) Kubelka and Munk (1931) formulated a law that deals with die thickness and 
density of pigment on a solid background. Beer's law describes the effect of concentration of a dye 
dispersed in a liquid on the transmittance of the liquid, or equivalently, the thickness of a series of 
transmitting filters. The laws all differ in their details, yet there are some themes common to both. 
The pigmentation laws describe a smooth transition from an unpigmented state to a state of saturated 
pigmentation. (Often, the unpigmented materials are white or grey, and darken as pigment density 
increases. Caucasians are usually pale in the winter, and darken gradually during the summer if 
allowed to frequent the beach.) More can be said about the smooth transition. Loosely speaking, 

7 '!Tie difference between pigment density changes and changes of materia! is important. P changes are changes in the 
density of a single pigment in a single sort of embedding material; A*/ changes are changes between two different 
pigments and (or) two different embedding materials. 
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the shape of the albedo function is preserved by the pigment density change. Two properties of the 
change of the albedo fimction make the notion above more precise. First, for a material characterized 
by Kubelka-Munk analysis or Beer's law, a change in pigment density affects the albedo function by 
either strictly increasing or decreasing its value at all wavelengths. Second, two albedo functions of 
the same material (but having different pigment densities) are related monotonically 8 . The conditions 
above will be called the normal pigmentation conditions, and pigmentation processes which obey them 
will be called normal 

Beer's law (Wyszecki & Stiles, 1973) will serve as an example of a pigmentation law. Suppose X 
and Y are two regions that transmit light through stacks of identical filters. Suppose further, that X 
and Y have the same surface orientation, and differ only in the number of filters in the stack. Then the 
image intensities Jx(A) and /y(X) that we could expect to measure are: 

MX) =IpWV x E*(\) 

MM =IpW) 1y e*(\) (4J 

^-v where p(h) is the transmittance of a thin filter, and 7* and 7y arc the number of such filters in regions 

X and y, respectively. Since the viewer-object-illuminant geometry is identical in X and y, the 
two multiplicative terms of the image intensity equation (1) that deal with scene geometry and do not 
appear in (4) can be considered "absorbed" as a constant factor in £*(X). 

Two facts about equations (4) warrant attention. First, a power relation is described between the 
reflectances of two regions of different pigment thicknesses. This is the simplest such pigment relation, 
and can be considered as a base from which more complex laws develop. Second, note that regardless 
of the function p(X), the first normal pigmentation condition holds. That is, [p{\)] lx > [p(X)] nrv \ 
for all X, or vice versa. Since the illuminant and scene geometry are identical in regions X and 
y, the normal pigmentation condition above is preserved in the available image intensities. Similar 
relationships between pairs of albedocs will hold for more complicated pigmentation laws (Kubelka & 
Munk, 1931), because such laws require monotonicity. 

4.5 Highlights 

The image intensity equation (1) only applies to matte surfaces. Highlight or specularity, a condi- 
tion when a surface acts as a partial mirror, is a common confounding process. In a highlighted region. 

8 Two single-valued functions / and g are related monotonkaly if f(x) > f(y) implies g(.r) > g(y). In particular, two 
monotonically related functions have their local minima and maxima at the same values in their domain. 
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image intensities are due to both a reflection of the light source, and a matte component due to the 
albedo of the underlying material. Appendix I shows that if most of the illumination of a region is 
due to the source that is reflected in the highlight, then the highlight # can be described as a process 
that strictly augments image intensities at all wavelengths. That is, if the normal highlight condition 
holds, the image intensity measurable in a highlighted region is the same as that of a neighboring 
matte region plus some (always positive-valued) function of wavelength 9 . Note this is identical to the 
first of the normal pigmentation conditions. Highlight differs from pigment density change in that it is 
always a positive change, and it does not necessarily preserve the shape of the image intensity function 
of wavelength from a neighboring matte regions. 



5 Details of Process Rejection: Spatial and Spectral Samples 

Our computational strategy is to examine image regions and to attempt to reject them as arising 
from the action of a confounding process on a single material. How might rejections be made? If a 
/-> process is lawful, as the major confounding processes are, then some system of equations characterizes 

the effects of the process. Given measurements of image intensities, if there is always a plausible 
solution 10 to the equations describing a process, then we can always mathematically interpret the 
image intensities as arising from that process. When we can always interpret image intensities as being 
due to a particular process, we can never reject the occurrence of that process. Whether or not there 
always exist plausible solutions depends on the number of samples being taken. In this section we 
examine the minimum information needed to be sometimes able to reject the presence of the major 
confounding processes. We will answer the questions: What is the minimum number of spatial and 
spectral samples needed to reject image intensities as arising from each of the confounding processes? 
How would increasing the number of samples increase the number of correct rejections? 

(Spectral samples will at first be assumed to be taken at a single wavelength. In Appendix II, it is 
shown that essentially any sort of spectral sample preserves the results of the following sections, which 
are derived using narrowband samples.) 

9 Since the word "highlight" denotes a region of greater brightness than surrounding regions, the normal highlight 
condition is reasonable and necessary to avoid mathematical anomalies such as "highlights" which are darker than 
surrounding regions. 

A plausible solution to a set of equations is one which assigns meaningful values to ph>sical variables. For example, 
a solution which assigns a number greater than one to a variable representing a reflectance is not physically meaningful 
and must be discarded 
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5.1 Rejecting the 5 Hypothesis 

Impossibility of Rejecting S with One Spectral Sample, Suppose one narrowband 
spectral sample is taken at some wavelength X 1# Two image intensities can then be measured: I iX and 
Ii Yi the intensities at \ x in image regions X and Y (see fig. 2). Can we reject the possibility that the 
only difference between regions X and Y is due to a shadow? Since either X or Y can be lit, and the 
other in shade, there are two equations to examine. In more formal terms, we need to know if we can 
rule out the existence of a plausible solution to either of two discrete versions of equations (2a): 

hx = {EiD + Eis)pi 

hr=E lDPl <*»> 

corresponding to X lit and Y shadowed, or 

hx — EidPi 

hY = (E iD +E ls )pi {5b) 

corresponding to X shadowed and Y lit. Note that p x is the albedo of the material at \ u and E iS and 
E\ D are the strengths of the synthetic and diffuse components of the illuminant. 

A shadow interpretation is some solution to equations (5a) or (5b) that assigns physically meaning- 
ful values to variables. If the measured values of I iX and I iY can be shown to be inconsistent with 
both equations (5a) and (5b), then there canot be a shadow interpretation. On the other hand, if any 
values of I iX and I iY yield a plausible solution to the equations, then there will always be a shadow 
interpretation. If there is always an S interpretation, rejection is impossible, and we must resort to a 
greater number of spectral samples. 

Intuitively, we are given two measurements of image intensity at only one wavelength. We are 
faced with the question: Is there any pair of image intensities that cannot be construed as arising from 
lit and shadowed portions of the same material? Given two measurements, one will be darker than the 
other. The darker region can be interpreted as being a shadowed continuation of the lighter region. So 
any pair of measurements can be construed as a shadow. Therefore, there can be no rejection of the 
shadow interpretation with only a single spectral sample. 

The casual argument above can be made rigorous. Is there ever a possibility that both equations 
r (5a) and (5b) will have no solution? First we'll consider solutions to equations (5a). One solution is 

given as follows: 



^^ RUBIN & RICHARDS 14 FINDING MATERIAL CHANGES WITH COLOR 



* _ T (6) 

hY 

where S is the parameter of the one-dimensional solution space of equations (5a). (Two equations 
in three variables almost always have a one-dimensional solution space.) This solution is acceptable 
or plausible as long as all the variables take positive values, and p { £ (0, 1). The latter constraint is 
satisfied by restricting 5 to the interval {h Y , oo). Note that£i D and pi are always positive, since 8 is 
restricted as above, and the measurement I 1Y is positive. Therefore all variables will take on positive 
values ifEis is positive, or, equivalently, if I iX > iiy. So whenever I iX > hr, there is a plausible 
solution to (5a). By the symmetry of equations (5b), it is clear that whenever h Y > hx, equations 
(5b) will have a plausible solution. Therefore, there will always be a solution to either (5a) or (5b), 
regardless of the values of measured image intensities. So there is always a shadow interpretation 
f^ for monochromatic measurements; the darker region can be construed as the shadowed one. Hence, 

monochromacy is inadequate to reject measured image intensities as arising from shadows 11 . 

Two Spectral Samples. 

Perhaps adding a second spectral sample will help. Will samples at two wavelengths from each of 
regions X and Y sometimes allow the rejection of the shadow hypothesis about the X-Y difference? 
Or will any sets of measurements ahvayshave a shadow interpretation, as in the monochromatic case? 

Intuitively, dichromacy should allow some rejections of the shadow hypothesis. Suppose that in the 
first spectral sample, region X is lighter than region Y. Then the same ought to hold for the second 
spectral sample if Y is to be construed as a shadowed continuation of the same material that composes 
X. So if one region is lighter in the first spectral sample, but darker in the second spectral sample, we 
probably aren't looking at a shadow. This follows from noting that in the case of shadow, both regions 
reflect a diffuse component of illumination, while the lit region has a synthetic component in addition. 

A more formal demonstration can be made, by rewriting the shadow equations (2a) in discrete 



11 The claim that monochromacy docs not allow shadow rejections only applies to the type of computations describee 
here. Of course there might be some type of achromatic computation, involving, say, texture, that would allow the 
rejection of shadow. 
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form for two spectral samples: 

hx = {Ew + Exs)p\ 

hx = {E2D + E2s)(>2 

hY=E M (7o) 

hy = E2DP2 

corresponding to X lit and Y shadowed, or 

hx = E iD px 

hx = E2DP2 

hy = {E lD + E xs )pi (76) 

hy = {E2D + E2s)p2 

corresponding to X shadowed and Y lit. 

Once again, the left-hand side terms are the measured image intensities, and the right-hand side 
contains the variables for which we attempt to solve the equations. Given any four measurements hx, 
hy, hx, and hr, will there always be a plausible solution to either equations (7a) or (7b)? 

f^s Consider first solutions to (7a). We have four equations in six unknowns, suggesting a two- 

parameter family of solutions. 

Below is a solution to equations (7a), parameterized by 6 and e: 

E\d = 8 

hy 
f» = ~ (8) 

Exs = s M_ l) 

HY 

E2S - e(£* - 1) 
hy 

The constraints of the values of the variables are once again that pi and /> 2 G (0, 1), and that all the 
variables take on positive values. So extending the results, of the previous section on a single spectral 
sample, it is clear that whenever both I xx > hy and hx > hy, there is a solution to equations (7a). 
Symmetrically, whenever both hy > hx and hy > hx, there is a solution to equations (7b). So 
there is a dichromatic rejection condition for shadows. If (hx > hy and I 2 y > hx) or (h y > hx 
and I 2 x > h)X there will be no plausible solution to equations (7a), nor to (7b), and the presence of 
shadow can be rejected. 
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Figure 4. Rejection of S by crosspoint in plots from regions X and Y of image 
intensity versus wavelength. 

There is a simple geometric interpretation of the dichromatic rejection condition: S can be rejected 
if the graphs of image intensity versus wavelength for regions X and Y intersect, or have a crosspoint. 
Furthermore, it has just been proved that the crosspoint condition is unique and minimal. It is minimal 
in the sense that it involves two spatial and two spectral samples; no smaller number of samples 
would do. It is unique in that, given two spatial and spectral samples, only the crosspoint condition 
holding among the four measurements can accurately lead to a rejection of S. Whenever there is no 
crosspoint, a shadow interpretation, possibly wrong 12 , can be found. A final point. Given two line 
segments (two spectral samples at two spatial locations yields two line segments in our system of rep- 
resentation) there is only one topological property that they have: intersection (or non-intersection). 
Fortunately, intersection, or crosspoint, has been shown to be a physically interesting condition in 
this problem, as well as a topological^ interesting one. Figure 4 illustrates the dichromatic rejection 
condition for 5. 

i2 A wrong interpretation is one that assigns values to physical variables that do not correspond to the actual values. 
Interpretations of sensory data are discussed in more detail in Richards et aL, 1981. 
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n-chromacy. It is easy to extend the dichromacy result above to the general case of n spectral 
samples (Xi, . . . , X n ). Imagine taking pairs of these spectral samples and plotting a line segment from 
region X and another segment from region Y. If any pair of spectral samples yields a crosspoint (the 
line segments intersect), then S can be rejected for the X-Y change. More formally, we can reject 
S n-chromatically if there exist j =^ k such that the line segment from I jX to I kX intersects the line 
segment from I jY to I kY - Equivalent^, we can reject S n-chromatically if there exist; 7^ k such that 
{Ijx-Ijy){lkx-Ikv)<0. 

5.3 Rejecting the 6 Hypothesis 

One Spectral Sample. The surface orientation change equations (3b) can be re-expressed as a 
single equation: 



^p**t^ 



Iy(\) fa W 

Equation (9) describes a simple proportionality holding over wavelength for image intensities. 
Clearly, if only a single spectral sample is taken at each of X and K, a trivial proportionality will hold 
between the measurements I iX and I iY , regardless of their values. (That is, for all measurements of 
image intensities, there are appropriate values of ft x and/3y so that a discrete version of equation (9) 
holds.) Hence a surface orientation change cannot be rejected monochromatically. 

Two Spectral Samples. Will a second spectral sample sometimes allow the rejection of image 
intensities as arising from a surface orientation change? And if so, under what conditions? Equation 
(9) can be rewritten in discrete form for two spectral samples: 

hx = (k ( } 

It is clear (by combining equations (10)) that wc can reject the 6 hypothesis whenever 



^ RUBIN & RICHARDS 18 FINDING MATERIAL CHANGES WITH COLOR 

Note that the nonproportionality condition above is narrower than the crosspoint condition that we 
derived for shadows; a crosspoint allows the rejection of both shadow and surface orientation change 
interpretations of image intensities. 

5.4 Rejecting the P Hypothesis 

In the preceding two sections, relationships among image intensities (at two spectral samples) 
were derived such that the discovery of the relationship could be taken as evidence that certain con- 
founding processes were not occurring in the image. The spectral crosspoint was the less strict of 
the two criteria derived. In this section, a different tack is taken. First, it is taken as obvious that 
monochromacy is inadequate to reject the presence of P. Next, it will be pointed out that a spectral 
crosspoint is good evidence that the image intensities under consideration do not arise from a change 
in pigment density 13 . It suffices to show (normal) pigmentation does not produce crosspoints. But this 
follows immediately from the first of the normal pigmentation conditions. Specifically, equations (4) 
can be rewritten for two spectral samples as 



/^N. 



hx _P2 

hr Pl Y 



ax (12) 



which implies (by the first normal pigmentation condition) that if f* > 1, then J f*- > 1, or vice 
versa. Therefore, if £* > 1 and fj < 1 (one of two possible spectral crosspoint conditions), then 
the presence of a normal pigmentation process can be rejected. 

5.5 Rejecting the H Hypothesis 

The presence of a spectral crosspoint can also be taken as evidence that no normal highlight is 
present. Since a (normal) highlighted region has greater image intensity at all wavelengths than a 
neighboring matte region, no spectral crosspoints can arise from such a pair of regions. (Also note that 
shadow changes are formally equivalent to highlights, in that lit and higlightcd regions correspond 
to striclty augmentative changes to image intensities in neighboring shadowed and matte regions, 
respectively.) Hence the spectral crosspoint allows die rejection of normal highlights. 

13 More precisely, it will be shown a crosspoint implies either (not P) or (/>, but not a norma! pigmentation process). 
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5.6 Rejecting Special Cases of S, 6, and H 

Shadows with Penumbrae. In the event of an extended source, shadows with penumbrae 
(Spen) can be rejected monochromatically (see Appendix II), but it should be noted that 
monochromatic rejection of the penumbra case requires three independent spatial samples, in con- 
trast to the two spatial samples needed for dichromatic rejection of S (point source). That is, for sharp 
(point-source-generated) shadows, two spatial and two spectral samples are the minimum needed 
for rejection. For shadows with penumbrae, three spatial samples at a single wavelength suffice for 
rejection. Nonmonolonicity of the discrete plot of image intensity versus postion in the image at any 
wavelength is sufficient for rejecting S pen . (Nonmonotonicity is when three collinear spatial samples 
arc neither strictly increasing or decreasing. This is identical to the condition that the center spatial 
sample of the collinear three has the greatest intensity, which cannot occur when the line passes 
through lit, penumbra, and shaded regions.) 

"Simple" surface orientation changes. A special class of 6 changes can be rejected with 
a single spectral sample taken over three spatial regions. Let O simple be the subset of 6 changes 
involving no inflections. O simp i e is similar to S pen in that both describe spatially monotonic processes. 
Clearly then, O sirnple can also be rejected monochromatically using three spatial regions with the 
nonmonotonicity test. (See Appendix II, which treats the shadow-with-penumbra case, S pen .) 

Highlights and Monchromacy. Consider three collinear samples passing through a normal 
highlight such that die center sample is taken in the highlight, and the end samples are taken in 
nearby matte regions of the same material. A luminance profile (at a single spectral sample) shaped 
like an upside-down vee can be expected for our piecewise linear representation. Therefore,' a 
monotonic luminance profile, or a vee-shaped profile, can allow the rejection of the possibility that the 
three samples correspond to matte-highlight-matte regions. ' 

5.7 Summary of Rejecting Individual Confounding Processes 

In the sections above, it was shown that a single spectral sample was not sufficient (except in 

special cases involving three spatial samples) to reject image intensities as the effect of a single major 

confounding process. Two spectral samples, however, were proven adequate to sometimes allow 

^ rejection of image intensities as the effect of a single process. The strictest rejection criterion occurred 

with surface orientation changes; nonproportionality was shown sufficient for rejection of 6. The 
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broadest criterion was the spectral cosspoint, derived from the shadow equations. Since shadow and 
(normal) highlight are similar in being processes that allow any strictly augmentative change over 
wavelength, the spectral crosspoint also allows the rejection of highlight. Furthermore, the crosspoint 
was shown to be the unique and minimal rejection criterion for 5. The crosspoint is a special case of 
nonproportionality, and hence allows the rejection of surface orientation changes as well. (And the 
crosspoint is more secure than the nonproportionality condition in situations of noisy measurements 
of image intensity.) Changes in pigment density are strictly augmentative (or subtractive) changes 
over wavelength, and therefore cannot cause crosspoints. So the crossoint is sufficient to reject P. 
Since each of the major confounding processes 5, H, 6, and P can be rejected dichromatically by the 
presence of a spectral crosspoint, the crosspoint criterion for rejection of single confounding processes 
is adopted. 

5.8 Rejecting Combinations of Processes 

The discussion above focused on what happens when a single confounding process occurs. But by 

/"■>. inspecting any photograph, it is obvious that arbitrarily chosen neighboring regions of the image may 

depict elements of the scene which differ in pigment density, surface orientation, and shadowing. (In 

particular, joint occurrences of O and S are common. Architecture provides examples in which one 

face of a polyhedral structure shades another.) 

How can the analysis be extended to cover instances of combinations of processes! Perhaps we 
can restrict our attention to some subset of neighboring image regions in which material-change- 
mimicking conspiracies among the confounding processes are unlikely. 

Which Neighboring Image Regions should be Examined?. It would be computationally 
exhausting to examine all possible neighboring image regions for material changes. And unnecessary 
as well. Material changes invariably produce with luminance discontinuities or edges in the image. If 
finding material changes is the goal being pursued, there's no point in looking for them where they're 
not going to be. That is, for the purpose of discovering material changes, "neighboring regions" 
should be taken to mean regions separated by edges. 

Can several confounding processes coincide at a single edge? As pointed out above, shadows and 
surface orientation changes can occur at a single edge. But aside from this case of self-shading;, it 
f"^ is generally believed that at an edge, a single process predominates (Marr & Hildrcth, 1980; Marr, 

1982). 
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(In the case of self-shading, no crosspoint can occur. By examining the shadow and surface orienta- 
tion change equations, it can be seen that a joint* occurrence of S and 6 is mathematically equivalent 
to an occurrence of 5 alone. And S changes cannot induce crosspoints. So self-shading cannot cause 
crosspoints.) 

5.9 Accepting M by Default 

When a crosspoint occurs across an edge, we know it does not arise from one of our confounding 
processes. If the reasoning above is correct, it is highly unlikely that such a crosspoint is caused by 
a peculiar conspiracy among 5, H, O, and P across an edge. It is worth asking if we have My 
captured the range of possible confounding processes by our models of 5, H, O, and P. All that could 
be said was that a highlighted region had greater image intensity at all wavelengths than a nearby 
matte region of the same material. The same is true for lit versus shadowed regions. And a slightly 
more stringent condition held for pigment density changes. It is clear from the discussion above that 
any process that acts on a scene variable(s) (that is a function of wavelength) by strictly increasing or 
/""N decreasing its value (at all wavelengths) can be rejected with the spectral crosspoint. Thus, when a 

spectral crosspoint occurs across an edge, it is almost always due to a material change. 

5.10 False Targets: Effectiveness of the Spectral Crosspoint in M Detection 

A word should be said about the theory of M detection presented here. If our characterization 
of confounding processes is accurate, most spectral crosspoints across edges will be M changes. 
Exceptions include instances of highlight and pigment density changes that aren't normal and rare 
(measure zero) coincidences of confounding processes at edges. So in signal detection language, the 
false target rate of crosspoints in M detection is low; most crosspoints will be material changes. 
As for the hit rate, it need not be the case that most M changes cause crosspoints in the image. 
Intuitively, only about half of the material changes in a scene will cause crosspoints. This should 
not be disturbing. The crosspoint computation is extremely easy, and provides immediate strong 
assertions. Of course a full theory of M detection is likely to involve other spectral computations, as 
well as nonspectral (textural) computations. 
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6 Relation to Psychophysics and Neurophysiology 

The theory we have presented begins with the physics underlying several scene processes affecting 
image intensities. Simple analysis of equations describing the processes has shown the crosspoint 
is the unique criterion for rejecting each of the confounding processes, and thus a good tool for 
spectrally identifying material changes. But what has this to do with color vision? The implica- 
tion of the results above is our crosspoint conjecture that biological visual systems interpret spectral 
crosspoints as material changes. 

Psychophysical Evidence. An interesting fact about human color vision is that it becomes un- 
stable under isoluminance conditions (Evans, 1948). Boundaries defined only by a chromatic change 
are weak; colors from one side of such a boundary are likely to invade the other. Color vision seems 
to require luminance discontinuities for stability. It was argued (section 5.8) that the only place in an 
image it makes sense to seek material changes is at edges. If color vision were concerned with the 
detection of material changes, then it would be an efficient system only if its computational resources 
/( *-* N were employed at edges. Besides, spectral computations carried out in arbitrary neighboring regions 

would be difficult, if not impossible, since many confounding processes could be acting together be- 
tween the two regions. The number of hypotheses to be entertained would become combinatorically 
nasty (that is, all possible subsets of S, H, 6, and P), and the biological value of enduring such 
tedium is dubious. 

Number of Spectral Channels. Biological color vision systems are usually di- or trichromatic 
(Walls, 1942). Why are there no pentachromatic systems, say? The theory suggests an answer. 
Dichromacy was shown sufficient to enable a system to make strong M assertions across edges. 
Adding a third spectral sample increases the chances of finding a crosspoint. That is, trichromacy 
seems to provide higher hit and lower miss rates in M detection than dichromacy. Why not add 
spectral channels ad infinitum! The spectral reflectances of natural objects arc almost always functions 
that change slowly over wavelength (Krinov, 1971). (There appear to be rarely more than three 
extremal points in reflectance function of wavelength.) It seems unlikely that a tetrachromatic system 
would detect enough additional crosspoints to be evolutionary advantageous. 

Comment on Physiology and Neurophysiology. 

f~^ Trying to design an operator to detect spectral crosspoints provides some insights into 

neurophysiology. Although biological photopigments are broadband (Dartnall,1962), Appendix [II 
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Figure 5. Designing an operator for detecting crosspoints. A) A crosspoint. B) A "Land unit," a circularly 
symmetric operator that compares image intensities at a single spectral sample (here short wavelength, or 5), 
over two spatial regions. Q The S+JE7L+ crosspoint detector. Two different Land units, S and L, are 
combined with a logical and across an edge, E. A nonzero response indicates a crosspoint. D) The S+EL+ 
unit resembles the double opponent unit See text 

shows that the particular shape of the photopigments used for taking spectral samples, and their 
degree of overlap, have no effect on the basic crosspoint finding. 

The crosspoint (fig. 5a) is equivalent to the following inequality among image intensities: 



which is identical to 



(hx-hy){hx-l2Y)<0 



(13a) 



{IlX-IlY){-hx + hY)>0 



(136) 



f*^- Suppose we desire a crosspoint operator that is circularly symmetric (for tessellation efficiency, say). 

Let the photopigments be S and L, having their respective X„ laj .'s at short and long wavelengths (see 
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fig. 5a). Equation (13b) can be rewritten: 

(Sx - S Y ){-L X + Ly) > (13c) 

where S x denotes image intensity in region X viewed through photopigment S, and so on. 

The operator depicted in fig. 5b is ideal for computing the (S x — S Y ) factor in equation (13c). 
This sort of unit we call a "Land unit" since it was suggested by Land's (1977) retinex theory of color 
vision. The Land unit has zero output in a homogeneous field, and maximum output when the center 
receives more energy at short wavelengths than the surround. 

Obviously, the firing of a single Land unit, even a maximal firing, does not provide information 
about the presence or absence of a crosspoint. A single unit can only inform about events at single 
spectral sample. Consider, however, the nonzero firing of a pair of Land units across an edge, as 
shown in fig. 5c. The S Land unit's nonzero firing implies S x > S Y \ the L Land unit's firing implies 
L Y > L x . The conjunction, or logical anding, of the two conditions over and edge provides a test 
for crosspoint. We propose, then, a crosspoint detector composed of the conjunction of two (spectrally 
f different) Land units on opposite sides of an edge. Note that without the requirement of an edge, this 

unit might respond to gradations of image intensities due to the effects of confounding processes. We 
call this operator an S+EL+ unit 14 . 

Two farther points should be raised about crosspoint detection with the SEL operator. First, the 
unit shown in fig. 5c only detects the type of crosspoint shown in fig. 5a. If Land units are constrained 
to have only positive response, as neurons are, a second type of unit, L+ES+, would be required to 
detect the other type of crosspoint {I\ X < I XY ). 

Second, a positive response of an on-center off-surround Land unit on one side of an edge implies 
a positive response of an off-center on-surround unit on the other side of the edge. In fig. 5c, imagine 
replacing the L+ unit on the Y side of the edge with an L~ unit (off-center) on the X side. Suppose 
the L"~ unit and an 5+ unit, both on the X side of an edge, were logically anded in a single unit. 
Such a unit might be sketched as in fig. 5d; it resembles a double-opponent cell, found in die retina 
of goldfish and in the primary visual cortex of primates (Daw, 1972) 15 . A final point should be 

14 This unit can be compared to Marr and Ullman's (1981) S+TS~ unit for directional selectivity. The two units are 
similar in that they have three components, the outer two of which are 'polar" forms of the same computation (S+ 
and S~ differ in sign of contrast; S+ and L+ are different spectral samples. Also, the center components of the two 
units perform more sophisticated computations than the ouier components. 

^"^ 15 ' 1 " nc unil depicted in fig. 5d may be misleading in suggesting an arithmetic sum of two land units, rather than a logical 

anding. Only detailed quantitative neurophysiologies!! stud) of such double- opponent units can reveal the computation 
they perform. 
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made about the double-opponent operator. It does not produce maximum output when it precisely 
straddles a edge with spectral crosspoint. The operator's largest response occurs when it is slightly 
offset from the material change boundary. This is reminiscent of properites of the V 2 G convolution 
operator 16 for luminance discontinuities (Marr & Hildreth, 1980). 



7 Conclusion 

Color vision systems evolved to solve certain problems in making sense of natural images. Natural 
images are complicated things, however, caused by a myriad of processes. While first intuition might 
be to study color vision over a simplified or restricted domain, we feel much can be understood by 
considering the more complicated natural domain. Certain regularities emerge from the confusion of 
the world, and these lawful relationships can be exploited as assumptions in the solution of sensory 
problems. Even given assumptions about the physics underlying scenes, natural images are still com- 
plicated. While ambitious goals can be successfully pursued in limited domains (Land, 1977), we 
feci that generally, more modest goals are appropriate to complicated domains. Our simple goal is to 
detect changes of material in an image using spectral information. 

One unusual characteristic of material changes is that they are unconstrained. Since an uncon- 
strained process cannot be sought directly (for what equations could we seek solutions?), we resort to 
inferring material changes by rejecting a class^ of processes that confound spectral image intensities. 
Rejecting the presence of a lawful process, it was pointed out, is often computationally much simpler 
then solving explicitly for the variables of the process. 

First, a computation based on a single spectral sample was sought that would allow, at least on 
some occasions, the rejection of the presence of one of the major confounding processes. But it 
was shown that no such computation exists (of the type discussed here); all neighboring pairs of 
regions look like lit and shaded portions of the same surface when viewed monochromatically. (It was 
shown, however, that special classes of confounding processes S pen and d 8imp i ei as well as H, could 
be rejected monochromatically using three spatial regions.) Next, it was found that adding a second 
spectral sample (for two spatial regions) did allow us to sometimes be able to reject the presence 
of a confounding processes. The spectral crosspoint was the broadest criterion derived, and it was 
shown that its discovery allowed the rejection of each of die confounding processes 5, //, 6, and P. 
fs, (Furthermore, the spectral crosspoint was proved the unique and minimal criterion for the rejection 

16 Thc laplacian of a two-dimensional gaussian. 
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of 5.) It was argued that by restricting the search for spectral crosspoints to edges, that complications 
arising from the joint occurrence of several confounding processes were unlikely. (And if several 
confounding processes did occur coincidental^, a crosspoint could only be generated under very 
unusual circumstances of illumination.) The spectral crosspoint allows us to say with great confidence 
that image intensities do not arise from S, H, 6, P, or any of a large class of possible confounding 
processes. The simultaneous rejection of this large class of processes allows us to infer with great 
confidence that a material change is taking place. Our crosspoint conjecture is that biological visual 
sytems interpret crosspoints as material changes. 

When no crosspoint occurs, more sophisticated computations are still possible (Richards, Rubin 
& Hoffman, 1981). The crosspoint strategy is not foolproof. While material assertions will be al- 
most always correct, there will be many missed material changes. Not all material changes produce 
spectral crosspoints. A good strategy for visual systems would be to locate the maximum absorption 
frequencies of their photopigments in order that a maximum number of crosspoints be detected 17 
. A study of the reflectances of natural objects could perhaps reveal if biological photopigments are 
^ located in wavelength in such a manner as to maximize the detection of crosspoints in an organism's 

environment. 

Finally, it was shown that an operator for the detection of crosspoints, constrained to use only 
simple arithmetic functions, to output only positive values, and to be circularly symmetrical, is the 
S + EL + unit, which performs a logical and of two spectrally different units across and edge. The 
SEL unit resembles the double-opponent cell commonly found in biological color vision systems. 

APPENDIX I 

Details about Highlights 

The image intensity equation (1) does not apply to highlight. Highlight occurs on certain surfaces 
when two conditions hold. First, the viewer direction, surface normal, and illuminant direction must 
be approximately coplanar. Second, <j> & 6\ that is, the angle of incident illumination must be nearly 
equal to the angle of reflection to the viewer. When the two conditions above hold, certain materials 
will display mirrorlike qualities, and the illumination source will be imaged on the surface. Let 
EsourctW be the source that is imaged in the highlight. We can expect the following image intensities 

Also, the photopigments should be located to minimize "doublecrossings," or situations in which a pair of complementary 
and cancelling crosspoints occur between the spectral sample points. Such situations would cause material changes to be 
missed. 
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from two neighboring regions of the same surface orientation that differ only in that one region has a 
highlight: 

Imattei^) = (E S ource(^) + E oih€r (\)) p(\) 

highlight^) =6E source {\) + (1 - 6){E 30urc€ (\) + E other (\))p{\) (15a) 

where 7 ma ^ e (X) and Ihi g hu g ht{^) are the image intensities expected from the matte and highlighted 
regions, and E other {\) represents all illumination other than the source E 80Urce {\) reflected in the 
highlight. Note in the equation for the highlighted region that there are matte and specular com- 
ponents (Evans, 1948; Horn, 1977) in some linear combination determined by fraction 6. The specular 
component is just some coefficient multiplied by the source; the albedo plays no role. The matte 
component involves all illumination, not just the direct source, as well as the albedo p{\) of the 
material. 

Next, note the highlight equation can be rewritten as follows: 



SS 



highlight^) = ^L(\) + ImatteW (156) 

where L(X) = E source (\) — [E 30urc€ (\) + E oth€r (\)]p(\). Therefore, the highlight becomes a purely 
additive process whenever L(\) > 0. We shall call this condition the nonnal highlight condition. It is 
equivalent to the following: 

£ J 8ource[*j ,»* 

E 80urce (\)+E other M > ™ (15c) 

for all X in the visually useful range. 

Since reflectances of surfaces usually have maxima around .7 (Krinov, 1971), the normal highlight 
criterion above is, loosely speaking, that the source provide a little more than twice the illumination 
than the diffuse light. Normal sunlit scenes will obey this criterion. 

APPENDIX II 

Shadows with Penumbrae (S p€n ): Rejection with One Spectral Sample 

Here we seek to rule out the existence of a solution to cither of the following discrete equations: 

I\,x = (Eis + E lD )p l 

h,Y = {aE ls + E iD )pi (16a) 

h,z=E lD pi 
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Figure 6. Rejecting S pC n by the nonmonolonicity of the discrete plot of image intensity 
versus spatial position at some wavelength. 

where X, Y and Z denote three collinear spatial regions being tested for their correspondence to lit, 
penumbra, and shaded regions, respectively, of a shadow with penumbra, or 



hx — Ei D pi 

hY = {oE iS + Ei D )pi 

hz = [Eis + E iD )pi 

corresponding to the other case that Z is the lit region and X is shaded. 
A solution to equations (16a), parameterized by 6 is given below; 

E lD = 6 
hz 



(166) 



P\ = 



Ex 



f~\_ 



is 



a = 



%x 



*\,z 

I\ t Y —h s Z 



i) 



(17) 



n,x- 



'i,z 
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The constraints are that all the variables be positive, and a G (0, 1). E iS is positive if £^ — 1 > 
Now the only interesting restriction is that a G (6, 1). 

a>0 implies h,y — li f z>0 implies h f y > I iiZ 
a < 1 implies h iX — h t z > h t y — h,z implies I i}X > h t y 

Therefore, any violation of the restriction I i)X > h,y > I ifZ implies the unsolvability of equa- 
tions (16a). By the symmetry of the alternate equations (16b), it is clear they provide the restriction 
h,z > h,y > hx- The pair of restrictions from (16a) and (16b) together imply that the three 
collinear image intensities must be either strictly increasing or strictly decreasing; that is, they must 
be monotonic. This rejection condition of nonmonotonicity is illustrated in figure 6, and can be 
summarized as follows: 

If the discrete plot of image intensity versus position across the S pen candidate is nonmonotonic, 
we can reject5 pen . Equivalent^, we can rejectS pen if (I lfX — I i)Y )(h f y — h,z) < 0. 

It is clear diat with additional spectral samples, the nonmonotonicity of the discrete plot of image 
intensity versus wavelength at any wavelength is sufficient to reject S pen 

However, in practice, there is a problem with the type of operator suggested by the theory: at what 
scale should triplets of spatial regions be examined for nonmonotonicity? Shadows-with-pcnumbrae 
can occur with large variations in spatial extent in images. Should a whole range of triplets be ex- 
amined? The computational complexity of such a task should make one wary of it. We take this scale 
problem to imply that, for practical use, the crosspoint condition is preferred. 

APPENDIX ni 

Broadband spectral samples cannot induce crosspoints. 

It will be shown that a crosspoint cannot arise in P, 6, or 5 situations from an overlapping of 
samples. 

1] Assume the change from X to Y is due to one of the confounding processes. Therefore, the 
continuous spectral energy distributions I x (\) and J r (X) do not intersect. (Otherwise the proof in 
section 5.1 could be shown wrong by taking samples straddling the intersection.) Assume, without loss 
of generality, that 7 A -(X) is always greater than 7y(X). 

f^) 2] Assume that our two spectral samples will be measured by "photopigments" P { and P 2 where 

each P { is simply a function mapping some wavelength interval into the unit interval (0, 1). 



^ m ^. 



j^ ma ™%^ 
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3] The sample measured at X,- in region X, S iX , and in region Y, S iY , are denned as follows. 

MX)P,(X)dX 
/•x <2 (18) 

S iY = A-(X)Pi(X)dX 

where \ i2 > \n and the interval (X a ,X i2 ) is the range of wavelengths over which the 
"photopigment" P { is sensitive. 

4] Six > S iY , for all i, follows directly from the fact that I X (K) > iy(X), for all X. 

5] Therefore, spectral crosspoints cannot be induced by overlapping spectral samples. 
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